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Millions Learning 
Real-time Scaling Lab 


overview 


Millions Learning,’ a project of the Center for 
Universal Education (CUE)? at the Brookings 
Institution, addresses the question of how to 
scale quality education for all children and 
youth. CUE is launching Real-time Scaling 
Labs in partnership with local institutions 
in a number of countries and US. cities to 
generate more evidence and provide prac- 
tical recommendations around the process 
of scaling in global education, encouraging 
a stronger link between research and prac- 
tice. These labs are not physical spaces, but 
rather a process established by CUE and 
partner institutions to learn from, support, 
and document existing efforts to scale in- 
terventions focused on improving children’s 
learning as they unfold in real-time. This will 
include observing, gathering, and analyzing 
data, as well as encouraging self-reflection, 
recommending course corrections based on 
existing evidence, documenting the scaling 
process in real-time, and sharing ongoing 
learning with those involved. The ultimate 
goal is to support initiatives as they scale 
while simultaneously gaining deeper insight 
into how policymakers, civil society, and 
the private sector can most effectively work 
together to bring about large-scale transfor- 
mation in the quality of children’s learning 
and their development. 


The impetus for the Real-time Scaling Labs— 
and overarching problem statement—is that 
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as we look at the education landscape of 
innovations working at a local level, it often 
does not translate to the systemic change 
needed to solve the global learning crisis. 
While there is growing evidence around 
which educational strategies improve chil- 
dren's learning, we know much less about 
how to translate this into improved policies 
and practices at scale. CUE believes that 
this requires a combination of technical and 
political strategies to scale evidence-based 
interventions, mechanisms to accelerate the 
adoption of new practices, and strengthen- 
ing local capacity for successful adaption 
and scale. 


Within and across each of the labs, CUE will 
draw lessons to better unpack and under- 
stand the scaling process, using these lessons 
to feedback directly into the initiatives partic- 


ipating in the labs, as well as contributing to Objectives of Real-time 
building a more robust, actionable evidence 


base on scaling globally. More specifically, Scaling Labs 


each of the labs will explore what technical 1. Strengthen scaling efforts through a 
and political considerations need to be ad- forum for peer-to-peer learning in which 
dressed, and how they have been addressed, lab participants discuss lessons learned 
to expand the impact of quality education and develop strategies to address chal- 
initiatives. This will include developing tools lenges faced during their education 
targeted for policymakers, practitioners, and interventions’ scaling journey. 

funders to offer practical guidance on how 

best to scale evidenced-based approaches 2. Provide ongoing evidence-based guid- 
in education. ance to lab participants, including 


practitioners and policymakers, on how 
to identify, adapt, and expand effective 
approaches to scaling education inter- 
ventions. Guidance will be drawn from 
the report Millions Learning: Scaling up 
quality education in developing coun- 


tries? and other scaling resources and 
tools in education and other disciplines. 

3. Document participants’ scaling expe- 
riences in real-time to feed back into a 
rapid and iterative learning cycle, as well 
as into the development of global public 
goods for scaling in education. 

4. Study modes of communication and 
collaboration among lab participants as 


a potential model for reflective learning 
and knowledge sharing generally. 


5. Identify mechanisms and approaches 
for connecting those innovating in edu- 
cation delivery with those designing and 
implementing policies and programs. 


6. Identify gaps in the scaling evidence 
base and areas for further research. 


DESIGNING AN ADAPTIVE LEARNING PROCESS TO SUPPORT LARGE-SCALE CHANGE IN EDUCATION 2 


Development of 
overarching lab 
methodology 


Background research on 
collective impact and 
adaptive implementation 


Research in the first stage of Millions Learn- 
ing identified that efforts to improve learn- 
ing outcomes at scale could benefit from a 
structured, neutral space for those involved 
to exchange knowledge and learn from 
one another about efforts to scale or affect 
large-systems change and to incorporate 
learnings into ongoing work and make nec- 
essary adjustments along the way.* 


CUE began designing plans for the Re- 
al-time Scaling Labs by conducting an ex- 
tensive literature review on collective impact 
and adaptive learning mechanisms, such as 
learning loops, innovation labs, hubs, accel- 
erators, and delivery teams. This was followed 
by a global scan of almost 200 existing 
mechanisms, spanning non-profit, govern- 
ment, private sector, and social enterprise 
initiatives across multiple sectors, including 
education, social policy, energy, technology, 
poverty reduction, and public health. The 
purpose of this literature review and scan of 
existing initiatives was to identify relevant 
theory, approaches, and specific initiatives to 
learn from and collaborate with, as well as to 
identify specific gaps where CUE could add 
value. 
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CUE found that there is a global burgeoning 
of mechanisms and infrastructure that bring 
together multi-sectoral actors to generate 
solutions to complex, social problems of a 
systemic nature, within technology, gover- 
nance, health, and other sectors. Despite 
their proliferation, Nowever, few of these 
collective impact mechanisms have a delib- 
erate focus on scaling, a clear area where the 
scaling labs can add value. Within the group 
examined, there is a large amount of diver- 
sity in terms of sector, focus, membership, 
funding, etc., suggesting that there is not 
one single blueprint or approach. However, 
a significant portion of the initiatives share 
core elements, in particular systemic think- 
ing, applied orientation, user-centered, and 
rapid iterative testing cycles. Finally, very few 
of these mechanisms had rigorous data on 
their effectiveness. 


Building off this work, CUE also conducted 
research on a wide range of methodologies 
and approaches, drawing not only from the 
principles commonly employed in existing 
collective impact mechanisms but also from 
continuous learning and adaptive program- 
ming more broadly. This included research 
into systems thinking, improvement science, 
Problem-Driven Iterative Adaptation (PDIA), 
change management, implementation 
science, and design thinking, amongst 
other approaches. Emerging research and 


practical experience suggest that the most 
effective approaches to addressing complex, 
developmental challenges, such as deliv- 
ering quality education, combine three key 
principles: problem-driven and_ politically 
informed; adaptive and entrepreneurial; and 
supporting change that reflects local realities 
and is locally led.5 A number of the method- 
ologies focus specifically on identifying and 
addressing root causes of complex systemic 
challenges, tailoring solutions to the local 
context, and utilizing experiential learning 
to adjust and make course corrections in 
real-time. 


Drawing from the existing mechanisms 
and relevant literature discussed above and 
informed by previous analysis undertaken 
for the report Millions Learning: Scaling up 


quality education in developing countries, 
input from policymakers, practitioners, 
funders, and researchers from around the 
world, and consultations in more than 10 
countries, CUE developed an overarching, 
flexible methodology for the Real-time Scal- 
ing Labs. More specifically, CUE compared 
and contrasted the above approaches as 
well as the key tenets of collective impact 
and applied learning mechanisms, identi- 
fied core principles, and adapted them into 
a methodology that aligns with the aims and 
context of the Millions Learning project and 
focuses more centrally on questions of scal- 
ing. Many of these methods and approaches 
have previously been applied to finding a 
solution to a problem, while the labs will 
focus on scaling a solution to a problem. 


Problem-driven and user-centered: The primary purpose of the scaling labs is to 
address pressing issues or bottlenecks that policymakers and practitioners face in 
scaling effective education interventions. To do this, it is critical that the process is 
focused on addressing issues identified by local stakeholders as high priority, rather 
than on implementing pre-determined solutions that do not address local priori- 
ties.° As discussed in PDIA, too often reformers import an external “best practice” 
intervention, without giving adequate consideration to the particulars of the local 
circumstances or to the underlying causes of the problem they are trying to solve, 
and then fail to see the results they hoped to achieve.’ In response to this, the lab 
approach will be problem-driven and user-centered, starting by identifying con- 
crete challenges to address that are deeply felt by the participants, led by the key 
local stakeholders who are close to the problem, understand it best, and are most 
incentivized to find a solution. Taking a problem-driven, user-centered approach 
can help control for what the Carnegie Foundation for the Advancement of Teach- 
ing calls “solutionitis’—the impulse to lead with a preconceived solution before fully 
understanding the problem to be solved.® 


Systems approach: The scaling labs will be grounded in systems thinking, or ana- 
lyzing systems as a whole as well as the interplay of relationships within them rath- 
er than examining the individual components independently. Drawing from the 
improvement science mantra that “every system is perfectly designed to achieve 
exactly the results it gets,"? one must understand the larger system if seeking to 


achieve different outcomes or results. Similarly, the scaling labs will begin by con- 
vening a diverse group of stakeholders to discuss and assess the nature of the prob- 
lem the intervention(s) seeks to address and the system that produces it. By taking 
a systems approach, participants will better understand and address the processes 
and mechanisms within the system that are hindering the type of improvement 
sought and to tackle the underlying causes behind the problems they are trying 
to solve. As Roy Steiner argues, “Simple solutions that don’t take a systems view 
into account often result in unintended negative consequences instead of durable 
change.”° 


Multi-stakeholder participation: Addressing complex challenges of a systemic na- 
ture requires the participation of actors across sectors and disciplines. The scaling 
labs recognize the importance of bringing together stakeholders with a diversity 
of perspectives and experiences in order to identify and address what are often 
structural barriers hindering scaling. As no one person can see the whole system, 
engaging a variety of different stakeholders with multiple perspectives helps to 
address the underlying causes within the larger system. 


Adaptive orientation: Today's rapidly changing and uncertain world requires adap- 
tive learners, institutions, and systems that readily learn, adjust, and evolve. The scal- 
ing labs recognize that just because an intervention has worked in one education 
system, does not mean it will have the same positive impact in every education 
system. Therefore, education reformers need to employ adaptive and innovative 
approaches to tailor solutions to fit in their own context and adapt global best prac- 
tices to local needs. Research and practical experience find that many interventions 
ultimately fail because they do not include mechanisms to address problems as they 
arise.” And given the complexity of most development challenges, there needs to 
be room for trial and error as well as flexible adaptation and course-correction.” Yet, 
rigid log frames and evaluation methodologies can prescribe solutions that leave 
little to no room for making adjustments when things start to go wrong. Instead, 
the Real-time Scaling Labs will experiment with an adaptive learning approach, 
employing an iterative process of testing, refining, and re-testing change ideas to 
leverage opportunities and address challenges to scaling. 


Quick feedback loops: Another critical component of this adaptive learning ap- 
proach are quick feedback loops, so that lab participants in the process of scaling 
interventions can test out ideas, fail quickly, reflect on the learning in real-time 
(rather than waiting for the end of a project), and use that information to inform 
decisions and adapt plans. The labs will employ cycles of testing, reflecting, and 
adapting with quick feedback loops built in, in order to test hypotheses and make 
changes throughout the process based on data and the group's evolving under- 
standing. This includes purposeful opportunities for reflection, which research in 
cognitive psychology and neuroscience has shown can lead to greater learning 
than simply acquiring more experience.® The lab methodology purposefully builds 
in routine moments for reflection on the scaling process, with the opportunity for 


lab participants to pause from the task of scaling in order to review data, reflect 
on what has already taken place and why, learn from what has happened thus far, 
and use that learning to adapt their work moving forward." This learning is a critical 
element of an adaptive approach." 


Political and technical considerations: Research and practical experience suggest 
that only tackling the technical challenges of expanding education access and 
quality is not enough; in order to provide millions of children with improved oppor- 
tunities to learn, it is also essential to understand and address the various factors 
outside of the classroom that determine whether an effective education innovation 
actually scales.'© As discussed in Millions Learning: Scaling up quality education 
in developing countries, it takes a combination of both technical and political 
strategies to implement effective initiatives at a large-scale.” Similarly, another core 
principle of the Real-time Scaling Labs is the focus on identifying and adapting 
education initiatives and pathways to scale that are politically, economically, finan- 
cially, and socially feasible within the particular local context. In the initial stage 
of the lab process, participants will develop or revise scaling plans that take the 
local political economy and context into account, and address both technical and 
political challenges to the scaling process. 


Input and feedback from 
experts and advisors 


Once the draft design of the scaling lab 
process was created, CUE solicited feedback 
from a wide variety of experts across disci- 
plines including education, health, agricul- 
ture, systems change, continuous learning, 
and adaptive programming (see Annex 1 
for a list of individuals consulted), to further 
refine and develop the lab methodology. 
Additionally, CUE held four in-person meet- 
ings with the Millions Learning high-level 
international advisory group, chaired by for- 
mer prime minister of Australia, chair of the 
Global Partnership for Education, and distin- 
guished fellow with the Center for Universal 
Education at Brookings, Hon. Julia Gillard, 
between March 2017 and May 2018. In these 
meetings, advisors discussed and provided 
feedback on plans for the scaling labs, in- 
cluding overall design, research approach, 
and internal and external communications 
strategies (see Annex 2 for a list of advisors). 


CUE incorporated key feedback received 
into various iterations of the lab approach 
and methodology, some of which included: 


* the importance of starting the process 
with clarity and consensus among lab 
participants on the nature of the prob- 
lem, as well as underscoring the reality 
that scaling does not automatically 
happen but is challenging and must be 
planned for from the start; 


* the need to give ample consideration 
to organizational capacity for scale and, 
therefore, the importance of institutional 
learning in the lab process; 


* the value of exploring opportunities to 
leverage in the scaling lab process in 
addition to examining problems or chal- 
lenges confronted; 


*« the importance of strong information 
management systems that allow for 


continuous learning balanced with the 
need for quick, simple data collection 
methods; 


* the need to build in enough time for 
iterations and reflections in terms of the 
overall timeline of lab process as well as 
the necessity to employ tight enough 
feedback loops in between lab conven- 
ings that allow for real-time learning; 


* the critical importance of identifying 
resources required to scale from the be- 
ginning, which will include considering 
budgets available and costs of the inter- 
vention as scale; and 


* and the necessity of ensuring the incen- 
tives to participate in the lab are clear by 
closely linking the learning and support 
provided through the process. 


CUE also presented the draft framework for 
input at international conferences, including 
at EAFIT University in Colombia in November 
2017, The Education University of Hong Kong 
in January 2018, Harvard University in Febru- 
ary 2018, and the Carnegie Foundation for 
the Advancement of Teaching’s Summit on 
Improvement in Education in San Francisco 
in April 2018. This inout was then incorporat- 
ed into a final version of the design. 


The resulting lab approach was designed 
to provide a common framework for all of 
the Real-time Scaling Labs, while also being 
flexible enough to be adapted to the local 
context. As such, for each of the Real-time 
Scaling Labs, the next step in the lab design 
process includes in-depth research into the 
local context as well as input and feedback 
from local stakeholders and partners to tailor 
the overarching lab approach to the particu- 
lars of each individual lab. 
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Recommendations on 
design of Real-time 


Scaling Labs 


Based on the background research and 
consultations held with key stakeholders 
as outlined previously, CUE proposes the 
following general design for the Real-time 
Scaling Labs to be tailored to each specific 
lab in partnership with local stakeholders. 


Overview 


Over the course of three years, the lab pro- 
cess will involve a series of in-person and 
virtual convenings and workshops that bring 
together policymakers, practitioners, and a 
diversity of other stakeholders around a par- 
ticular education initiative or set of initiatives 
that is in the process of scaling. 


« Initial convenings will focus on articulat- 
ing a shared scaling goal for the inter- 
vention and determining the initiative’s 
current place along a scaling pathway, 
along with identifying opportunities 
and obstacles currently confronted in 
achieving those scaling goals and prog- 
ress towards scale. 


* The lab will then support a process for 
developing and/or refining a scaling 
action plan that addresses root causes 
of the scaling challenges identified and 
leverages opportunities to accelerate 
scaling, drawing from the Millions Learn- 
ing framework and the scaling literature. 


« Participants will put scaling plans into 
practice and reconvene periodically for 
reflection periods, to assess data col- 
lected on short-term targets, determine 
what elements should be adjusted, 
analyze what lessons have been learned 
thus far, and evaluate progress along the 
scaling pathway towards the end goal. 


* Throughout the three years, the lab will 
employ an iterative process of testing 
and refining change ideas to strength- 
en scaling efforts, which will allow par- 
ticipants to learn from what works and 
what does not work, make adjustments 
to scaling plans, and ultimately progress 
along the scaling pathways identified 
while also improving capacity to scale. 


As such, the lab will provide a reflective, neu- 
tral space for stakeholders to discuss shared 
challenges and opportunities faced in scal- 
ing an education intervention, discover the 
underlying causes of those challenges, and 
identify and adapt approaches based on 
data collected to address these issues, ulti- 
mately contributing to improving learning 
outcomes for children and youth. 


Within this lab process, each individual 
scaling lab will focus primarily on its own 
education initiative(s) and the challenges 
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Define scaling pathway 


and opportunities it faces in bringing that 
initiative to scale. However, CUE will also fos- 
ter opportunities for cross communication, 
knowledge sharing, and peer-to-peer learn- 
ing between the labs, through virtual meet- 
ings, webinars, and occasional in-person 
convenings. Additionally, CUE is interested in 
going deeper into questions that emerged 
through research conducted for the Millions 
Learning study and subsequent consulta- 
tions around the world. Beyond testing the 
Millions Learning framework and exploring 
the various scaling pathways and approach- 
es within each lab, CUE has also identified 
two key scaling challenges that each of the 
labs will explore in depth. The first focuses on 
education alliances, drawing lessons around 
mobilizing diverse actors from public, private, 
and social sectors to align around shared 
incentives and objectives of improving learn- 
ing outcomes. The second is around flexible 
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and end goal 


Identify scaling 
challenges to address 
& opportunities to 
leverage 


£ 


adaptation, looking at the critical question of 
adapting and scaling an effective approach 
across contexts. 


Throughout the project, CUE will observe 
the process of implementing, adapting, and 
scaling the selected interventions via quan- 
titative and qualitative methods, analyze 
data on an ongoing basis to identify lessons 
learned and challenges confronted during 
the scale up, leverage existing tools and 
expertise to help address challenges as they 
arise, and recommend course corrections 
identified through the iterative learning pro- 
cess as well as existing evidence and expe- 
rience from other labs. CUE will document 
the entire process undertaken by the various 
actors—from conception to design to imple- 
mentation to monitoring to outcome—so 
that others can continue to build upon what 
is learned through each scaling lab. Ongo- 


ing lessons learned will be shared between 
the Real-time Scaling Labs in each location 
through in-person and virtual exchanges, 
including two global convenings that will 
bring together representatives from all of 
the scaling labs twice during the three-year 
period. 


At the end of the project, CUE will produce a 
policy report that tells the story of the design, 
implementation, adaptation, and scaling of 
the intervention(s) and broader efforts to col- 
lectively strengthen education ecosystems. 


tors to achieve large-scale systems change. 
Additionally, the publication will provide rec- 
ommendations about how best to foster the 
transformational change urgently needed in 
education—identifying mechanisms and ap- 
proaches that connect those innovating in 
education delivery with those designing and 
implementing policies, in conjunction with 
evidence about scaling, and affecting broad 
systems change. CUE will also develop tools 
targeted for policymakers, practitioners, and 
funders to offer practical guidance on how 
best to scale evidenced-based approaches. 


It will also serve as a model of collaborative 
learning and collective action, including 
what strategies can be used to develop a 
coherent approach engaging multiple ac- 


Tools (not 
exhaustive) 


Pre-Lab Activities 


Tailor general 
lab method- 
ology and 
approach to 
local context 
in collabora- 
tion with local 
partners 


CUE will work with partners to adapt the general lab approach to the 
local context. This will include a series of calls and meetings with lab 
stakeholders and partners to get ongoing feedback on design of lab 
process, tailor materials to meet participants’ expectations and needs, 
and build consensus and buy-in prior to official launch of the lab. 


- Generic step- 
by-step lab 
approach 


Hire in-coun- 
try facilitator 


CUE and the local partner institution will hire two local, part-time 
consultants to work on the ground throughout the lab process. The 


and docu- documenter will be responsible for capturing the details and data of 
menter how the initiatives scaled and how the lab approach functioned and 

supported the scaling process, while the facilitator will be responsible 

for guiding the in-person and virtual convenings. 
Conduct CUE will conduct background research to inform each scaling lab, - Stakeholder 
background which may include landscape analysis, stakeholder mapping, scoping mapping 
research in pieces on particular issues, etc. - Landscape 
support of the analysis 
lab process 


- Political econ- 
omy analysis 
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1 


Select lab 
participants 


Clarify the 
education 
problem and 
intervention 
that will be 
the focus of 
the lab 


Global 
facilitator & 
documenter 
training 


CUE and local partner institution will identify and invite approximately 
20-25 key stakeholders involved in scaling the selected initiative, 
representing a diversity of sectors and perspectives (including represen- 
tatives from government, civil society, private sector, etc.), to participate 
in the lab process. This will include a mix of high-level representatives 
along with those directly responsible for implementation of policies 
and programs. Ideally the same individuals will participate in the entire 
three year process. 


While the overall lab process will be problem-driven, focused on 
identifying and addressing challenges faced in the process of scaling, 
the starting point for each lab will be to identify an evidenced-based 
education initiative (or set of initiatives) in the process of scaling that 
the lab will focus on and to ensure that all key stakeholders engaged in 
the lab have a common understanding of the nature and scope of the 
problem that the intervention(s) seeks to address. 


CUE will organize a multi-day, in-person training of facilitators and 
documenters involved with each of the Real-time Scaling Labs. Training 
will provide the opportunity to become familiar with lab design, re- 
search protocol, methodology, communications and data platform, and 
data collection tools, ensuring consistent and systemic data collection, 
analysis, and facilitation of lab convenings. 


Tools (not 
exhaustive) 


- Stakeholder 


mapping 


- Microsoft 


Teams 


- Millions Learn- 


ing framework, 
scaling lab 
survey, scaling 
rubric, and 
other data 
collection 
instruments 


- Lab step-by- 


step approach 


Initial Lab Convening - 2-3 day workshop, hosted by local partner institution and co-designed and co-led 


by CUE, local partner, and in-country facilitator 


Administer 
question- 
naire to lab 
participants 
on scaling 
knowledge 
and survey on 
expectations 
for participat- 
ing in lab 


Lab facilitator will administer questionnaire on scaling knowledge to all 
lab participants to gather baseline data in order to assess any increase 
in scaling knowledge over the course of the lab process. A similar ques- 
tionnaire will also be administered to a sample of non-participants at 
the baseline, midline, and endline to serve as comparison with the lab 
group as well as capture any eventual spillover of knowledge to broader 
community. An additional survey will include questions on participants’ 
expectations, what they hope to learn from the lab process, and how 
would they define success at the end of the three years. 
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- Millions 


Learning 
scaling lab and 
questionnaire 
survey 


Tools (not 
exhaustive) 


Overview 

of scaling 
definitions, 
frameworks, 
and best 
practices 


Define 
scaling goals 
and assess 
intervention’s 
place along 
scaling 
pathway 


Identify 
scaling 
challenges to 
address and 
opportunities 
to leverage 


Brief presentation and discussion of key scaling definitions, frammeworks, 


pathways, best practices, and challenges. 
- Why is scaling so difficult? 


- Defining scale: What are the various definitions of “scaling impact”? 
- Pathways to scale: What are the various pathways and approaches 


to scale? 


Presentation of Millions Learning 14 core ingredients and scaling 
framework 
- How does this resonant with current work? 


Quick review of the education intervention(s) being scaled and the 
nature and scope of the problem it addresses. 


Define scaling goal. 


- What exactly are we scaling? What is the end goal? What does 
success look like in 1-2 years? 5-7 years? 


Identify current scaling approach being pursued. 


- Is intervention scaling vertically, horizontally, functionally, and/or 
organizationally? 


Assess intervention’s progress along scaling pathway. 
- What stage of scaling is the intervention at? 


Conduct scalability assessment. 


Discuss scaling drivers, such as Millions Learning 14 core ingredients, and 
spaces, such as fiscal, political, and cultural spaces, and importance of 


applying systems thinking. 


Examine existing bottlenecks inhibiting scaling and potential opportu- 


nities to accelerate progress. 


- What is causing this challenge or bottleneck? 
- What is leading to this opportunity? 


- What strengths and weaknesses does the scalability assessment 
reveal? 


Identify concrete, actionable constraints or opportunities to address 
through the Real-time Scaling Lab. 


- Millions Learn- 
ing framework 


- Management 
Systems Inter- 
national (MSI) 
Scaling Up 
Management 
Framework 


- Hartmann and 
Linn spaces 
and drivers 
framework 


- ExpandNet 
9 Step Guide 
for developing 
a scaling-up 
strategy 


- Millions 
Learning 
scaling rubric 


- International 
Development 
Innovation 
Alliance (IDIA) 
Architecture 
for Scaling 
Innovation 


- Millions Learn- 
ing framework 


- MSI Scalability 
Assessment 
Tool (SAT) 
Checklist 

- Landscape 
analysis 

- Fishbone 
diagram 

- Driver diagram 

- Process 
mapping 

- System 
mapping 
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Tools (not 
exhaustive) 


Conduct or- 
ganizational/ 
institutional 
assessment 


Develop a 
scaling action 
plan 


Identify key stakeholders to be engaged in scaling action plan, includ- 
ing roles, influence, and interest. 


Conduct an organizational/institutional assessment in regard to the 
scaling strategy to identify gaps in human resources, infrastructure, 
incentives, or systems required to implement the scaling strategy and 
operate at scale. 


Lab participants, led by facilitator, draft scaling action plan, using a 
shared format and guided by examples. This will include the following: 


Decide which key drivers to tackle first, based on stakeholder priorities, 
scalability assessment, ability to enact change, cost analysis, etc. 


Identify change ideas" to be tested that participants hypothesize will 
address the identified underlying causes of the scaling challenge and 
advance progress along scaling pathway. 


- What changes might we introduce to address the root causes 
identified? 

- What does relevant theory and research suggest? What seems 
plausible to practitioners? To policymakers? What addresses the 
systemic problems we face? 


- What is financially, politically, and socially feasible in the local 
context? 


Discuss plans for implementing change ideas, to be refined throughout 


the first iteration. 


- What specific change ideas will be tested? 

- What specific actions will be taken to enact these change ideas? 
- What is the timeframe for this iteration? 

- What outcomes are expected? 


Iteration One - Approximately 3 months 


- MSI Scalability 
Assessment 
Tool (SAT) 
Checklist 


- Millions Learn- 
ing scaling 
action plan 
framework 


- MSI Scalability 
Assessment 
Tool (SAT) 
Checklist 


- ExpandNet 
9 Step Guide 
for developing 
a scaling-up 
strategy 

- Hartmann and 
Linn spaces 
and drivers 
framework 

- NESTA 
Strategies for 
Scaling Social 
Innovation 

- J-PAL Gen- 
eralizability 
Framework 


Draft scaling 
action plan 
for iteration 
and validation 


Lab facilitator drafts scaling action plan based on discussions during 
initial lab convening. Draft is shared with lab participants for feedback 
and iteration. 


CUE conducts background research as needed to inform the scaling 
plan, such as costing exercise, organizational assessment, etc. 
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- Millions Learn- 
ing scaling 
action plan 
framework 


Tools (not 
exhaustive) 


Reflection Convening One - One day workshop hosted by local partner institution and co-led by CUE and 
in-country facilitator 


Review and 
validate 
scaling action 
plan 


Develop data 
collection and 
rapid learning 
strategy 


Bring stakeholders together to discuss drafted scaling action plan, - Millions Learn- 
recognizing the development of the plan is only a first step and it will ing scaling 
remain a living document used to track progress and make ongoing action plan 
adjustments. framework 

; ; ' - MSI Scalability 
- Reevaluate and adjust if necessary scaling pathway De eeernent 
- Conduct scalability assessment of new plan Tool (SAT) 
- Validate plan among key stakeholders Checklist 


Lab participants simplify and refine intervention as needed, based on 
scalability assessment, scaling action plan, and best practices. 


Design simple, feasible measures to assess progress on the plan that 
can be implemented and used quickly by non-experts. Measures 
depend on the problem identified and solutions being tested. 


- How will we know if the changes introduced are leading to improve- 
ment??? 

- How will we measure progress towards the end goal? Towards 
addressing the underlying causes? 

- How will we measure if change ideas are being implemented as 
planned? 

- How will we be sure that the change ideas are not creating unin- 
tended consequences? 


Agree on data collection approaches/requirements for documenter 
during next iteration period. 


Agree on cross-pilot learning and communications strategies during the 
next iteration period and clarify role facilitator will play. 
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Tools (not 
exhaustive) 


Iteration Two - Approximately 6 months 


Implement Lab participants will implement scaling action plan, supported by the - Millions Learn- 
scalingaction _ facilitator, including researching unanswered questions, negotiating ing scaling 
plan unresolved issues, and testing change ideas. action plan 
framework 
Lab facilitator and documenter, with support from CUE and local Millions 
partner institution, will: Learning 


- Collect data against simple targets identified in previous convening scaling rubric 


- Provide regular updates analyzing data captured, presenting key 
achievements or milestones attained and critical bottlenecks faced 


- Conduct individualized follow up with key stakeholders 
- Hold targeted discussions around specific issues 


- Microsoft 
Teams 


- Conduct targeted research to fill information gaps 


Documenter will capture and share key progress and obstacles con- 


fronted. 
Ist Millions CUE will organize the first of two global convenings that will bring 
Learning together key stakeholders from the initial set of the Millions Learning 
Real-time Real-time Scaling Labs (approximately 5-7 labs) to discuss the overarch- 
Scaling ing lab design and approach, shared research questions and protocol, 
Lab Global anticipated challenges to consider, and communication channels for 
Convening sharing learning and data, among other issues. Additionally, it will be an 


opportunity for stakeholders to become familiar with the larger universe 
of scaling labs, make connections, and identify opportunities for poten- 
tial collaboration in the years ahead. 


Reflection Convening Two - One day workshop hosted by local partner and co-led by CUE, local partner, 


and facilitator 


Reflection Bring stakeholders together to analyze results of second iteration. - Millions Learn- 
workshop ing scaling 

- What worked and did not work? action plan 

- What assumptions did we make? framework 

- What lessons did we learn? - Millions 


Learning 


- What changes need to be made to the scaling plan? ; . 
scaling rubric 


- What adjustments should be made to the interventions? 
- Data gathered 


- Do we need to reevaluate the scaling pathway? during itera- 
- How did the process of collecting, analyzing, and communicating tion period 
learning in real-time work? - Expandnet 
- How did communications between stakeholders work? Implementa- 
tion Mapping 
Check in with government on its current priorities and needs. Tool 
- MSI Scalability 
Adapt scaling action plan for iteration three. Conduct scalability assess- Assessment 
ment of new plan. Tool (SAT) 
Checklist 
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Tools (not 


exhaustive) 


Over the course of three years, the Real-time Scaling Lab continues to follow cycles of six-month iteration 
periods to implement scaling action plan and collect data against short-term targets identified, followed by 
in-person reflection convenings with lab participants to analyze results and make necessary adjustments to 
scaling action plans and change ideas to test, on the way to achieving the short and long-term goals and 
progressing along the scaling pathway. At the midline and endline convenings, CUE will administer a follow 
up survey to lab participants on their scaling knowledge, to assess whether it has increased through the lab 
process, as well as on their perceptions of the lab’s effectiveness and whether it met their expectations. 


2nd Millions 
Learning 
Real-time 
Scaling 

Lab Global 
Convening 


CUE will organize the second global convening that will bring together 
key stakeholders from each of the Millions Learning Real-time Scaling 
Labs to share key insights and learning, discuss experiences to date 
with the lab process, identify opportunities to collaborate, and plans for 
sustaining the labs beyond the initial phase. 


Final documentation and dissemination 


Development 
of policy brief, 
targeted 
tools, and oth- 
er multimedia 
products 


National and 
international 
report and 
tool launch 
and global 
dissemination 
of findings 


CUE will produce a seminal report that tells the story of the design, 
implementation, adaptation, and scaling of intervention(s). CUE will 
also develop tools targeted for policymakers, practitioners, and funders 
to offer practical guidance and recommendations on how best to scale 
evidenced-based approaches that improve learning. These tools will be 
designed for use by a wide audience. 


To accompany the report and tools, CUE will also produce other knowl- 
edge sharing and dissemination products including blogs, videos, briefs, 
and podcasts throughout the lab process. 


CUE and local partner institution will hold national event(s) to launch 
final report(s) and tools, bringing together lab participants and other 
key stakeholders. In addition to the national launch event, CUE may 
host other global sharing and dissemination events with key stakehold- 
ers including policymakers, practitioners, funders, and academics, in 
Washington D.C., New York City, and other locations. 
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Initial Lab 
Convening 


Final Convening 
and Report 
Launch 


4th Reflection 
Convening 


3rd Reflection 
Convening 


2nd Reflection 
Convening 


Ist Reflection 
Convening 


Facilitator supports development, implementation, and adaption of scaling action plan, documenter records process & collects data 


6 MONTHS 


6 MONTHS 6 MONTHS 


Documenter & Facilitator Training: 
* Hold multi-day training of all lab 


consultants initial set of labs 


* Focus on lab design, research protocol, * Discuss lab design, shared research 
questions and protocol, anticipated 
challenges, and communication 


methodology, communications 
platform, and tools 
channels 


Research focus of the 
scaling labs 


Each of the labs will provide an opportuni- 
ty to test and refine the Millions Learning 
scaling framework and 14 core ingredients 
identified (see Annex 4). This will include 
looking at the issue of costs in each of the 
labs, including how costs of interventions 
are considered from the start, how unit costs 
change when delivered at scale, and where 
cost efficiencies or economies of scale can be 
achieved. Beyond testing the Millions Learn- 
ing framework and exploring various scaling 
pathways and approaches, in each lab CUE 
is interested in examining more deeply two 
issues that have emerged through research 
conducted for the Millions Learning study 
and subsequent consultations as key chal- 
lenges facing policymakers, funders, and 
practitioners in their efforts to scale quality 
education interventions. Both issues go 
deeper into some of the “core ingredients’ 
identified in the Millions Learning study as 
key drivers contributing to the scaling pro- 
cess. 


17 MILLIONS LEARNING REAL-TIME SCALING LABS 


1st Global Lab Convening: 
* Bring together key stakeholders from * Bring together key lab stakeholders 


6 MONTHS 6 MONTHS 6 MONTHS 


2nd Global Lab Convening: 


to share key insights and learning, 
experiences to date with process, 


for sustaining labs 


1. Education alliances (partnerships): The 
labs will draw lessons around mobilizing 
diverse actors from public, private, and social 
sectors to align around shared incentives 
and objectives of improving learning out- 
comes. The need to engage various partners 
to deliver quality-learning interventions at 
scale presents both opportunities and chal- 
lenges. While partnerships are not always 
effective, they have been essential in a vari- 
ety of contexts to bring together the diverse 
skills, resources, and assets required to scale 
sustainably and equitably. 


Within the labs, this may include looking at 
the following issues: 


* Role of intermediaries in bringing to- 
gether various systems actors, including 
questions of sustainability and eventual 
handover of intermediary's role (examin- 
ing the “black box” between small-scale 
learning innovations and achieving 
large-scale implementation); 


collaboration opportunities, and plans 


* Role of financing in resourcing the scal- 
ing of learning approaches, in particular, 
innovative financing arrangements such 
as Public-Private Partnerships, taxes lev- 
ied, flexible, long-term financing, etc.; 


- Strategies to engage the government 
and maintain impact, particularly where 
the innovation has been initiated by non- 
state actors; and 


* Role of accountability among the various 
actors. 


2. Flexible adaptation: The labs will also look 
at the critical question of adapting effective 
learning interventions across contexts, which 
requires striking the right balance between 
contextualization and maintaining fidelity to 
the original model. 


Within the labs, this may include looking at 
the following issues: 


« Identifying the core elements of the in- 
terventions responsible for driving gains 
in learning that should be replicated 
across contexts, while contextualizing 
the rest to local circumstances; 


* Developing a deeper understanding of 
the process of contextualizing learning 
approaches, particularly in resource-poor 
environments; 


« Understanding and addressing the po- 
litical economy factors that arise when 
scaling effective learning approaches 
(particularly as scaling from an NGO-led 
initiative to government-led); 


* Reconciling conflicts between donors’ 
and host governments’ accountabili- 
ty-driven needs for ex-ante targets and 
work plans with the expectation that 
successful scaling requires course-cor- 
rections and a degree of flexibility; 


* Role of data and the systems required to 
collect, analyze, learn, and share informa- 
tion as initiatives are adapted and scaled 
in new contexts; and 


« Issues of capacity, human resources, and 
skills needed to innovate and implement 
quality education initiatives at small- 
scale versus operational complexities of 
implementing at large-scale. 


Criteria for lab selection 


CUE developed criteria for selecting lab 
partners and initiatives, balancing an inter- 
est in comparability and the development 
of a strong research base with the realities 
of where there exists demand for a lab, 
partners with aligned interests, evidence of 
effectiveness, and government engagement. 
Criteria for scaling lab selection include: 


* Demand-driven, where there is strong 
interest from a local partner; 


¢ Strategic timing, where there is a reform 
process underway or other catalyst to 
leverage; 


« Evidence of the effectiveness of the inter- 
vention being scaled; 


* Committed local partner with aligned 
interests and capacity to engage; 


¢ Political will and buy-in from government 
partners; and 


e Addresses a critical issue in education 
facing many countries. 


When selecting the labs, CUE deliberately 
sought diversity across the cohort, in terms 
of geographic location, intervention of focus, 
scaling pathway (i.e., horizontal, vertical, etc.) 
and approach (i.e., replication, adaptation, 
etc.), as well as the role of state and non-state 


DESIGNING AN ADAPTIVE LEARNING PROCESS TO SUPPORT LARGE-SCALE CHANGE IN EDUCATION 18 


actors in designing, delivering, and financ- 
ing the intervention. At the same time, the 
development of a common framework and 
shared research questions for all the labs will 
allow for cross-comparability. 


Lab roles and participants 


In each location, CUE will co-design and 
jointly implement the Real-time Scaling 
Labs with a local partner institution, who will 
also play a role in documentation, analysis, 
and dissemination of findings. Local partners 
include academic institutions, non-govern- 
mental organizations, government agencies, 
and philanthropic institutions. While the 
exact structure of each lab may vary, the 
local partners will all co-design and serve as 
the point of contact between CUE and other 
key stakeholders; convene the meetings and 
workshops; co-manage local consultants 
along with CUE; contribute to the analysis 
and documentation process; and contribute 
to dissemination, especially at the national 
and local levels. 


Each scaling lab will also contract or work 
with two part-time individuals to work on- 
the-ground throughout the lab process. 
These individuals may be current govern- 
ment employees or local researchers in order 
to help build local capacity and institutional- 
ize scaling knowledge. The facilitator will be 
responsible for co-designing and co-facilitat- 
ing the initial lab workshop and subsequent 
in-person and virtual lab convenings over the 
three years, as well as supporting lab partic- 
ipants during the iteration cycles to draft, 
implement, test, and adapt scaling action 
plans. The facilitator will also play a role in 
sharing real-time learning rapidly with lab 
participants. The documenter will serve as 
researcher and documenter for each scaling 
lab and will be responsible for capturing 
the details of how the scaling lab approach 
functioned and how it supported the scaling 
process; “telling the story” of how the educa- 
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tion intervention scaled; and sharing lessons 
learned within and across the scaling labs. 
The documenter will gather data during lab 
convenings and throughout each iteration 
period, tracking progress towards short- and 
long-term targets identified during the lab 
convenings, and report findings periodically 
to CUE and the local partner. 


The Real-time Scaling Lab process itself will 
bring together key stakeholders involved in 
the implementation and scaling of the se- 
lected education intervention or set of inter- 
ventions, representing a diversity of sectors 
and perspectives (including government, 
implementers, civil society, the private sector, 
etc.), to participate in a series of convenings 
and workshops over the three-year period. 
This will include a mix of high-level represen- 
tatives along with those directly responsible 
for the implementation of policies and pro- 
grams. Participant diversity is an essential 
component of the scaling lab, as it ensures 
perspectives from a wide variety of view- 
points, resulting in stronger problem analysis 
and helping the group to avoid relying on 
their own assumptions and preconceptions. 
Lab participants will be selected by CUE and 
the local partner, based on the local context 
and specifics of the education intervention 
for each lab. Key to the lab success will be 
participant buy-in in the process and active 
participation in the convenings. Ideally, the 
same individuals would participate in the 
entire three-year lab process. 


Communications tools 


An essential aspect of the labs is that infor- 
mation is gathered and analyzed in real-time 
and key insights are shared on a regular basis 
in order to facilitate learning and inform nec- 
essary adaptations. In each lab, the plan is for 
an on-the-ground documenter to collect this 
information during in-person convenings 
and throughout the six-month iteration pe- 
riods, while CUE, in collaboration with local 


partners, will analyze, synthesize, and share 
emerging findings with lab stakeholders on 
an on-going basis. 


CUE researched a wide variety of platforms 
and tools for communication, data collec- 
tion, document sharing, and peer-to-peer 
exchange within and between the Real-time 
Scaling Labs that would be appropriate for 
low-resource environments. In this research, 
CUE sought functionality in three catego- 
ries: Communications, data collection and 
storage, and knowledge management. First, 
in terms of communications, CUE wanted 
a tool that would allow and encourage lab 
participants (in particular the on-the-ground 
documenter and facilitator in each lab) to 
communicate easily and frequently with 
CUE and with each other within and across 
labs, through real-time chatting as well as 
voice and video calls. Specifically, CUE was 
interested in a platform that would encour- 
age discussion and peer-to-peer sharing, 
replicating the informality and real-time 
conversation of a WhatsApp group. For data 
collection and storage, CUE wanted a plat- 
form that would allow the on-the-ground 
documenter and facilitator for each lab to 
regularly upload their data and reporting, 
store it in a shared location, and have the 
research easily available to CUE and local lab 
partners. For the knowledge management 
component, CUE sought a platform that 
would store and archive relevant documents 
and conversations between lab participants 
in order to support the documentation of 
the lab process and record how it happened 
in real-time. 


Other requirements for the platform in- 
cluded: availability of the platform on both 
a desktop and phone; at least some of the 
services available with a slower internet con- 
nection; worldwide availability; affordability; 
and ease of use. Finally, rather than having 
lab participants use multiple tools and pro- 
grams for these varied services, CUE ideally 
wanted a single platform that brought all of 


these functions together, in order to stream- 
liné communications, ensure no data or in- 
formation gets lost, and make participation 
as straightforward as possible. 


After extensive research, CUE has selected 
Microsoft Teams as the best platform for 
communications and data sharing within 
the Real-time Scaling Labs. Microsoft Teams 
is a chat-based workspace within Office 365 
that is deeply integrated with the other Mic- 
rosoft Office products. Given that Brookings 
already uses the Office 365 suite, Teams 
fits in seamlessly with the Millions Learning 
workflow and is already directly synced with 
the software, programs, and file manage- 
ment systems Millions Learning uses (such 
as SharePoint, OneDrive, etc,). 


Using Teams, lab participants (in particular 
the on-the-ground documenters and facili- 
tators) can chat, call, or video conference in 
groups or one-on-one, enabling real-time 
and ongoing collaboration and commu- 
nication on specific projects and_ tasks. 
Conversations within Teams are divided into 
dedicated channels to enable focused con- 
versation and exchange, such as the “Philip- 
pines” channel and the “Monitoring & Eval- 
uation” channel. All conversations in Teams 
are archived and searchable, automatically 
documenting all of the communication tak- 
ing place and allowing CUE to go back and 
revisit important conversations later in the 
research and writing process. 


Within Teams, participants can also upload 
and share files, co-edit documents, share 
screens during presentations and video 
calls, host large webinars, schedule and fa- 
cilitate team meetings, take notes, and hold 
phone conversations (with phone numbers 
available for 400+ cities around the world 
and voicemail and translation functions). 
Teams seamlessly integrates into the other 
Office 365 products (such as Word, Power- 
Point, Excel, Skype, and OneDrive) as well as 
most third-party services (such as Twitter), 
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essentially allowing all of the work for the 
Real-time Scaling Labs to happen in one 
place. The creation of a “Millions Learning 
Team” also creates a larger “Group,” which ef- 
fectively serves as a hub for all of the Team's 
work. This includes a share point site, where 
all Team documents are stored in the cloud, 
and a shared Team email address, calendar, 
and to-do list. Lab documenters and facilita- 
tors will be able to upload their research and 
documentation directly to this site, so that 
all of the information is stored on the cloud 
in a single place accessible from anywhere. 
CUE will be able to control permissions to 
the various folders, and so can decide who 
has access to what information within and 
across labs. 


In summary, through Teams lab consultants 
and partners can actively participate in re- 
al-time learning, sharing, communication, 
and collaboration. Individuals can have 
focused conversations on a particular topic 
within one channel, while another general 
channel keeps all lab participants informed 
of critical news and updates. Documentation 
and data from the labs can be uploaded to 
the group files, so that data is all stored in a 
single place and backed up by Office 365. As 
the owner of the group, CUE will have access 
to all communications and folders and can 
assign permissions and control access. Teams 
is available as both a desktop and mobile 
app and lab consultants and participants 
can have free access to the service through 
Brookings’ guest access. CUE will create and 
manage the Team and will train all lab con- 
sultants on how to use the program during 
the global consultant training to ensure un- 
derstanding and correct usage. 
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Measures of success 


CUE is in the process of designing a com- 
mon data collection and monitoring and 
evaluation system across all of the Real-time 
Scaling Labs. This remains a work-in-progress 
but to date CUE is considering collecting 
data at two levels: 


At the lab process level, CUE will explore 
two fundamental questions. 


1. How did the lab process function? 


2. Are individual labs making progress to- 
wards their internal targets within each 
iteration? 


To answer the first, CUE will measure how the 
lab functioned through data collected by an 
on-the-ground documenter in each country. 
This consultant will be documenting the en- 
tire lab process, including measures on how 
the lab functioned, such as participation in 
twice-yearly convenings, attrition, respon- 
siveness of participants, etc. Additionally, 
the documenter will gather survey data at 
the baseline, midline, and endline of the lab 
process on participants’ expectations of the 
lab, satisfaction toward meeting these ex- 
pectations, and impressions of the impact of 
the lab process. These measures and surveys 
will be the same across all of the scaling labs. 


CUE will also be collecting data around 
scaling plans developed during the lab con- 
venings and measuring progress towards 
meeting those targets. At the initial conven- 
ing for each scaling lab and in the first itera- 
tion that follows, lab participants will jointly 
develop a scaling action plan that they will 
continue to iterate on during the lab process, 
including ideas they want to test to acceler- 
ate progress towards their scaling goals and 
short-term targets they aim to achieve. At 
each lab convening, participants will identify 
which of these ideas and targets they wish 
to focus on in the subsequent six-month 


iteration period as well as quick measures 
to assess progress towards these targets. The 
on-the-ground documenter will then gather 
data based on these measures throughout 
each iteration period, which will be used to 
inform real-time learning and adaptation. 
Lab participants will review and discuss the 
data gathered at each reflection convening, 
and adjust the scaling action plan, including 
the short-term targets, based on insights 
gathered. 


At the lab outcome level, CUE is interested 
in understanding if, how, and to what extent 
the labs have contributed to the expected 
outcomes of Real-time Scaling Lab, which 
are: 


1. Increased knowledge of how to scale 
quality education initiatives, specifically 
among the policymakers, practitioners, 
and funders participating in the lab 
process, as well as spillover effects in the 
broader community. 


2. Demonstrated progress towards scale for 
the initiatives within the individual labs. 


3. Development and uptake of an adapt- 
able, replicable model to support 
peer-to-peer learning and knowledge 
exchange for scaling in education. 


To measure the first outcome—an increase 
in knowledge—CUE will administer a ques- 
tionnaire on scaling knowledge to lab 
participants at the baseline, midline, and 
endline of the lab process in order to gather 
quantitative data on any increase in scal- 
ing knowledge over the course of the lab 
process. CUE will also administer the same 
questionnaire at the same points to a sim- 
ilar group of non-participants to serve as a 
comparison. This will include a question on 
where non-participants have learned about 
scaling, in an attempt to capture whether 
their colleagues who are participating in the 
labs are sharing their learning. CUE is consid- 


ering other methods by which to measure 
how knowledge gained through the lab 
process might be spilling over to the broader 
community as well.2° 


To measure the second outcome, CUE will 
track progress towards achieving the ulti- 
mate scaling goals collectively identified in 
each lab. As described above, targets will 
be identified in initial lab convenings and 
tracked by the in-country documenter, along 
with local partners, and discussed during 
reflection convenings. 


To test the hypothesis that the Millions 
Learning 14 core ingredients contribute to 
the process of scaling, CUE is considering the 
development of a rubric, or scoring guide, to 
measure the strength of each of the inter- 
ventions vis-a-vis each of the core ingredients 
at baseline, midline, and endline of the labs. 
This rubric will be standardized across all of 
the scaling labs, with the recognition that 
some core ingredients may not be applicable 
in every situation. Data to inform the rubric 
will be gathered both through observation 
as well as interviews with key stakeholders. 


To measure the third outcome—develop- 
ment and uptake of an iterative learning 
model to support scaling—CUE will develop 
a checklist of steps in the development 
of the lab model and assess progress to- 
wards meeting them, such as iteration on 
a dynamic theory of change. CUE will also 
incorporate data gathered on how the 
lab functioned into the assessment of this 
outcome. Additionally, CUE will look at the 
uptake of this model, through metrics such 
as citations and requests for guidance and 
speaking engagements. 


For each of these metrics, the on-the-ground 
documenters connected with each lab will 
collect data during the lab convenings and 
throughout each iteration cycle. Each lab 
documenter will use a set of questions and 
measurement tools developed by CUE and 
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local partners that are common acrossall labs 
to help ensure data standardization, quality, 
and comparability. The documenters will up- 
load and share data collected routinely with 
CUE and local partner institutions, who will 
analyze the data within and across the labs 
on an ongoing basis in order to draw out and 
synthesize key learning and identify issues to 
investigate and discuss with lab participants. 
The in-country facilitator will also play a role 
in sharing this learning with lab participants 
(in addition to the bi-yearly convenings) to 
inform adjustments to implementation and 
scaling plans. 
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Annex 3: Millions Learning 
Real-time Scaling Lab theory 


of change 


Activity 
Accompany a range 
of education interven- 
tions in the process of 
scaling in order to: 


1. Learn from the 
process in real-time 


2. Support efforts to 
scale by providing 
feedback and practical 
recommendations 
around scaling process 


3. Document process 


Output 1 


Generate new 
evidence and develop 
tools for policymak- 
ers, practitioners, 

and funders that 
provide recommen- 
dations on scaling 
evidence-based 
approaches 


Intermediate 
Outcome 1 


Increased knowledge 
of how to scale quality 
education initiatives 
among lab partici- 
pants 


Long-term 
Outcome 1 


Deeper understanding 
among policymakers, 
practitioners, and 
funders of principles 
and enabling condi- 
tions behind scaling 


Stee ec enenceceneecesencececeesasecseseceseececsens 


Long-term 
Outcome 2 
Increased local capac- 


rae ity among policymak- 
within and across labs >——S}A—__—_ 25 diel Clee OMEIS 


to inform global policy 
and practice 
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Output 2 


Create an adaptable, 
replicable approach to 
support peer-to-peer 
learning and knowl- 
edge exchange on 
scaling in education 


Intermediate 
Outcome 2 


Initiatives within labs 
demonstrate progress 
towards scale 


for scaling impact in 
education 


Prreeretrer etree rer etrecerereerereretrerererererg) 


Long-term 
Outcome 3 


Uptake of adaptable 
model to support 
peer-to-peer learning 
and knowledge 
exchange on scaling in 
education 
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Annex 4: Millions Learning 14 
core ingredients 


Cost-effective Elevating 


DESIGN learning 


teachers 


Local education Flexible adaptation 


needs 
Learning champions Windows of 
DELIVERY and leaders 


opportunity 


Education Technological Better data 


alliances advances 


Long-term education 


FINANCE financing 


Flexible education “Middle phase” 


financing financing 
ENABLING Supportive policy 


ENVIRONMENT environment 


A culture 
of R&D 
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